Differential diagnosis checklists reduce diagnostic error differentially: a randomized experiment

Introduction Wrong and missed diagnoses contribute substantially to medical error. Can a prompt to generate alternative diagnoses (prompt) or a differential diagnosis checklist (DDXC) increase diagnostic accuracy? How do these interventions affect the diagnostic process and self-monitoring? Methods Advanced medical students (N = 90) were randomly assigned to one of four conditions to complete six computer-based patient cases: group 1 (prompt) was instructed to write down all diagnoses they considered while acquiring diagnostic test results and to finally rank them. Groups 2 and 3 received the same instruction plus a list of 17 differential diagnoses for the chief complaint of the patient. For half of the cases, the DDXC contained the correct diagnosis (DDXC+), and for the other half, it did not (DDXC−; counterbalanced). Group 4 (control) was only instructed to indicate their final diagnosis. Mixed-effects models were used to analyse results. Results Students using a DDXC that contained the correct diagnosis had better diagnostic accuracy, mean (standard deviation), 0.75 (0.44), compared to controls without a checklist, 0.49 (0.50), P < 0.001, but those using a DDXC that did not contain the correct diagnosis did slightly worse, 0.43 (0.50), P = 0.602. The number and relevance of diagnostic tests acquired were not affected by condition, nor was self-monitoring. However, participants spent more time on a case in the DDXC−, 4:20 min (2:36), P ≤ 0.001, and DDXC+ condition, 3:52 min (2:09), than in the control condition, 2:59 min (1:44), P ≤ 0.001. Discussion Being provided a list of possible diagnoses improves diagnostic accuracy compared with a prompt to create a differential diagnosis list, if the provided list contains the correct diagnosis. However, being provided a diagnosis list without the correct diagnosis did not improve and might have slightly reduced diagnostic accuracy. Interventions neither affected information gathering nor self-monitoring

a prospective observational study

Introduction Emergency rooms (ERs) generally assign a preliminary diagnosis to patients, who are then hospitalised and may subsequently experience a change in their lead diagnosis (cDx). In ERs, the cDx rate varies from around 15% to more than 50%. Among the most frequent reasons for diagnostic errors are cognitive slips, which mostly result from faulty data synthesis. Furthermore, physicians have been repeatedly found to be poor self-assessors and to be overconfident in the quality of their diagnosis, which limits their ability to improve. Therefore, some of the clinically most relevant research questions concern how diagnostic decisions are made, what determines their quality and what can be done to improve them. Research that addresses these questions is, however, still rare. In particular, field studies that allow for generalising findings from controlled experimental settings are lacking. The ER, with its high throughput and its many simultaneous visits, is perfectly suited for the study of factors contributing to diagnostic error. With this study, we aim to identify factors that allow prediction of an ER's diagnostic performance. Knowledge of these factors as well as of their relative importance allows for the development of organisational, medical and educational strategies to improve the diagnostic performance of ERs. Methods and analysis We will conduct a field study by collecting diagnostic decision data, physician confidence and a number of influencing factors in a real-world setting to model real-world diagnostic decisions and investigate the adequacy, validity and informativeness of physician confidence in these decisions. We will specifically collect data on patient, physician and encounter factors as predictors of the dependent variables. Statistical methods will include analysis of variance and a linear mixed-effects model. Ethics and dissemination The Bern ethics committee approved the study under KEK Number 197/15. Results will be published in peer-reviewed scientific medical journals. Authorship will be determined according to ICMJE guidelines. Trial registration number The study protocol Version 1.0 from 17 May 2015 is registered in the Inselspital Research Database Information System (IRDIS) and with the IRB (‘Kantonale Ethikkomission’) Bern under KEK Number 197/15

Is language an issue? Accuracy of the German computerized diagnostic decision support system ISABEL and cross-validation with the English counterpart.

OBJECTIVES Existing computerized diagnostic decision support tools (CDDS) accurately return possible differential diagnoses (DDx) based on the clinical information provided. The German versions of the CDDS tools for clinicians (Isabel Pro) and patients (Isabel Symptom Checker) from ISABEL Healthcare have not been validated yet. METHODS We entered clinical features of 50 patient vignettes taken from an emergency medical text book and 50 real cases with a confirmed diagnosis derived from the electronic health record (EHR) of a large academic Swiss emergency room into the German versions of Isabel Pro and Isabel Symptom Checker. We analysed the proportion of DDx lists that included the correct diagnosis. RESULTS Isabel Pro and Symptom Checker provided the correct diagnosis in 82 and 71 % of the cases, respectively. Overall, the correct diagnosis was ranked in 71 , 61 and 37 % of the cases within the top 20, 10 and 3 of the provided DDx when using Isabel Pro. In general, accuracy was higher with vignettes than ED cases, i.e. listed the correct diagnosis more often (non-significant) and ranked the diagnosis significantly more often within the top 20, 10 and 3. On average, 38 ± 4.5 DDx were provided by Isabel Pro and Symptom Checker. CONCLUSIONS The German versions of Isabel achieved a somewhat lower accuracy compared to previous studies of the English version. The accuracy decreases substantially when the position in the suggested DDx list is taken into account. Whether Isabel Pro is accurate enough to improve diagnostic quality in clinical ED routine needs further investigation

What factors affect team members' evaluation of collaboration in medical teams?

INTRODUCTION Perceived teamwork quality is associated with numerous work-related outcomes, ranging from team effectiveness to job satisfaction. This study explored what situational and stable factors affect the perceived quality of teamwork during a specific team task: when a medical team comprising a senior (supervisor) and a junior (trainee) physician diagnoses a patient. METHODS During a field study in an emergency department, multisource data describing the patients, the diagnosing physicians, and the context were collected, including physicians' ratings of their teamwork. The relationships between perceived teamwork quality and situational (e.g., workload) and stable (e.g., seniority) factors were estimated in a latent regression model using the structural equation modeling (SEM) approach. RESULTS Across the N = 495 patients included, SEM analyses revealed that the patient-specific case clarity and urgency influenced the perceived teamwork quality positively, whereas the work experience of the supervisor influenced the perceived teamwork quality of both supervisor and trainee negatively, albeit to different degrees. DISCUSSION Our findings shed light on the complex underpinnings of perceived teamwork quality, a performance-relevant factor that may influence work and organizational effectiveness in healthcare settings

Frequency and predictors of unspecific medical diagnoses in the emergency department: a prospective observational study.

BACKGROUND Misdiagnosis is a major public health problem, causing increased morbidity and mortality. In the busy setting of an emergency department (ED) patients are diagnosed under difficult circumstances. As a consequence, the ED diagnosis at hospital admittance may often be a descriptive diagnosis, such as "decreased general condition". Our objective was to determine in how far patients with such an unspecific ED diagnosis differ from patients with a specific ED diagnosis and whether they experience a worse outcome. METHODS We conducted a prospective observational study in Bern university hospital in Switzerland for all adult non-trauma patients admitted to any internal medicine ward from August 15th 2015 to December 7th 2015. Unspecific ED diagnoses were defined through the clinical classification software for ICD-10 by two outcome assessors. As outcome parameters, we assessed in-hospital mortality and length of hospital stay. RESULTS Six hundred eighty six consecutive patients were included. Unspecific diagnoses were identified in 100 (14.6%) of all consultations. Patients receiving an unspecific diagnosis at ED discharge were significantly more often women (56.0% vs. 43.9%, p = 0.024), presented more often with a non-specific complaint (34% vs. 21%, p = 0.004), were less often demonstrating an abnormal heart rate (5.0% vs. 12.5%, p = 0.03), and less often on antibiotics (32.0% vs. 49.0%, p = 0.002). Apart from these, no studied drug intake, laboratory or clinical data including change in diagnosis was associated significantly with an unspecific diagnosis. Unspecific diagnoses were neither associated with in-hospital mortality in multivariable analysis (OR = 1.74, 95% CI: 0.60-5.04; p = 0.305) adjusted for relevant confounders nor with length of hospital stay (GMR = 0.87, 95% CI: 0.23-3.32; p = 0.840). CONCLUSIONS Women and patients with non-specific presenting complaints and no abnormal heart rate are at risk of receiving unspecific ED diagnoses that do not allow for targeted treatment, discharge and prognosis. This study did not find an effect of such diagnoses on length of hospital stay nor in-hospital mortality

The Utility of an Online Forward Triage Tool During the SARS-CoV-2 Pandemic: Health Care Provider and Health Authority Perspectives.

Introduction The SARS CoV-2 pandemic poses major challenges not only to patients but also to health care professionals and policy-makers, with rapidly changing, sometimes complex, recommendations, and guidelines to the population. Online forward triage tools (OFTT) got a major boost from the pandemic as they helped with the implementation and monitoring of recommendations. Methods A multiphase mixed method sequential explanatory study design was employed. Quantitative data were collected first and informed the qualitative interview guides. Video interviews were held with key informants (health care providers and health authorities) between 2 September and 10 December 2020. Audio-recordings were transcribed verbatim, coded thematically and compared with patient perspectives (framework). Objectives To explore the perspectives of health care providers and authorities in Canton Bern on the utility of a COVID-19 OFTT, as well as elicit recommendations for telehealth in future. Results The following themes emerged; (i) accessibility (ii) health system burden reduction (iii) utility in preventing onward transmission (iv) utility in allaying fear and anxiety (v) medical decision-making utility (vi) utility as information source (vii) utility in planning and systems thinking. The health care providers and health authorities further provided insights on potential barriers and facilitators of telehealth in future. Conclusion Similar to patients, health care providers acknowledge the potential and utility of the COVID-19 OFTT particularly as an information source and in reducing the health system burden. Data privacy, doctor-patient relationship, resistance to change, regulatory, and mandate issues, and lack of systems thinking were revealed as barriers to COVID-19 OFTT utility

Automated identification of diagnostic labelling errors in medicine

Objectives: Identification of diagnostic error is complex and mostly relies on expert ratings, a severely limited procedure. We developed a system that allows to automatically identify diagnostic labelling error from diagnoses coded according to the international classification of diseases (ICD), often available as routine health care data. Methods: The system developed (index test) was validated against rater based classifications taken from three previous studies of diagnostic labeling error (reference standard). The system compares pairs of diagnoses through calculation of their distance within the ICD taxonomy. Calculation is based on four different algorithms. To assess the concordance between index test and reference standard, we calculated the area under the receiver operating characteristics curve (AUROC) and corresponding confidence intervals. Analysis were conducted overall and separately per algorithm and type of available dataset. Results: Diagnoses of 1,127 cases were analyzed. Raters previously classified 24.58% of cases as diagnostic labelling errors (ranging from 12.3 to 87.2% in the three datasets). AUROC ranged between 0.821 and 0.837 overall, depending on the algorithm used to calculate the index test (95% CIs ranging from 0.8 to 0.86). Analyzed per type of dataset separately, the highest AUROC was 0.924 (95% CI 0.887-0.962). Conclusions: The trigger system to automatically identify diagnostic labeling error from routine health care data performs excellent, and is unaffected by the reference standards' limitations. It is however only applicable to cases with pairs of diagnoses, of which one must be more accurate or otherwise superior than the other, reflecting a prevalent definition of a diagnostic labeling error

Patient safety culture in a university hospital emergency department in Switzerland - a survey study.

Aim of the study: Poor safety culture, bad teamwork, non-functional inter-departmental working relationships and increased cognitive demands are associated with higher amounts of adverse events in hospitals. To improve patient safety, one of the first steps is to assess safety culture among health care providers in an institution. Considering the sparsity of studies addressing patient safety culture in Europe and Switzerland, the aim of the present study was to assess patient safety culture in the emergency department of a University Hospital in Switzerland. Methods: We employed the Hospital Survey On Patient Safety Culture, developed by the U.S. Agency for Healthcare Research and Quality. 140 questionnaires were distributed to nurses and physicians. Two weeks after the first questionnaire, we performed a sensitization campaign addressed to health care providers, and then repeated the survey. We calculated composite scores for each question category and percentages of positive responses for each dimension. For group comparisons such as possible differences relating to education and duration of employment and to compare results of the first and second survey we used T-tests. The results were compared to other published surveys outside of Switzerland. Results: Particularly positive assessments were found for the categories "nonpunitive response to errors", "teamwork within units", "supervisor/manager expectations and actions promoting patient safety" and, compared to other hospitals, also "staffing". The lowest average percent positive responses were found in the categories "frequencies of reported event", "teamwork across units" and "handoffs and transitions". Nurses and health care personnel with a longer employment history had an overall more negative assessment of patient safety culture, when compared to physicians and personnel with a shorter duration of employment, respectively. Conclusions: The present study has identified strengths and potential weaknesses in the safety culture of a large university hospital emergency department in Switzerland. The results provide opportunities for improvement of patient safety in particular in the reporting of adverse events, in interaction across units and patient transitions. Furthermore, as we employed a standardized self-assessment tool similar to previously published studies, the work contributes to the establishment of a benchmark for hospital safety culture at the national, European and international level

A Utility Framework for COVID-19 Online Forward Triage Tools: A Swiss Telehealth Case Study.

The SARS-CoV-2 pandemic caused a surge in online tools commonly known as symptom checkers. The purpose of these symptom checkers was mostly to reduce the health system burden by providing worried people with testing criteria, where to test and how to self-care. Technical, usability and organizational challenges with regard to online forward triage tools have also been reported. Very few of these online forward triage tools have been evaluated. Evidence for decision frameworks may be of particular value in a pandemic setting where time frames are restricted, uncertainties are ubiquitous and the evidence base is changing rapidly. The objective was to develop a framework to evaluate the utility of COVID-19 online forward triage tools. The development of the online forward triage tool utility framework was conducted in three phases. The process was guided by the socio-ecological framework for adherence that states that patient (individual), societal and broader structural factors affect adherence to the tool. In a further step, pragmatic incorporation of themes on the utility of online forward triage tools that emerged from our study as well as from the literature was performed. Seven criteria emerged; tool accessibility, reliability as an information source, medical decision-making aid, allaying fear and anxiety, health system burden reduction, onward forward transmission reduction and systems thinking (usefulness in capacity building, planning and resource allocation, e.g., tests and personal protective equipment). This framework is intended to be a starting point and a generic tool that can be adapted to other online forward triage tools beyond COVID-19. A COVID-19 online forward triage tool meeting all seven criteria can be regarded as fit for purpose. How useful an OFTT is depends on its context and purpose